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Using Hierarchical Linear Modelling (HLM), this study aimed to identify factors 
such as ESL/ELL/EAL status that would predict students' reading performance 
in an English language arts exam taken across Canada. Using data from the 2007 
administration of the Pan-Canadian Assessment Program (PCAP) along with 
the accompanying surveysfor students and the schools, a two-level (student level 
and school level) HLM model was analyzed for predictive relationships. Results 
showed that, at the student level, predictors such as students' participation in 
class discussions, language spoken at home, parents' encouragement to read at 
a young age, and the number of individual projects requiring students to work 
outside of class contributed significantly to the students' reading scores. However, 
none of the school-level predictors were found to be significant. All the significant 
predictors contributed to only 12% of the variability in this HLM model. Iden¬ 
tification of more significant variables is needed in order to have a full picture of 
students' reading competence and achievement. 

S'appuyant sur la modelisation lineaire hierarchique (MLH), cette etude porte 
sur l'identification desfacteurs, comme le statut ALS/ELL/ALA, qui prediraient 
les acquis en lecture d'eleves lors d'un examen d'anglais administre partout au 
Canada. Les auteures ont employe des donnees du Programme pancanadien d'eva¬ 
luation (PPCE), y compris les sondages connexes pour les eleves et les ecoles, afin 
d'analyser les liens predictifs d'un modele HLM a deux niveaux (eleve et ecole). 
Les resultats indiquent que les predicteurs tels la participation des eleves aux dis¬ 
cussions en classe, la langue parlee a la maison, la mesure dans laquelle les parents 
encouragent leurs enfants a lire des un jeune age et le nombre de projets indivi- 
duels exigeant du travail a I'exterieur de la salle de classe, contribuaient defagon 
significative aux resultats des eleves en lecture. Toutefois, aucun des predicteurs 
au niveau de Vecole ne s’est revele comme etant significatif. Dans leur ensemble, 
les predicteurs significatifs n'ont contribue qu'd 12% de la variability du modele 
MLH. Afin d'arriver a une vue globale du rendement et de la competence en lec¬ 
ture des eleves, ilfaudra identifier plus de variables significatifs. 


In Canada, the ever-changing population landscape has a direct impact on 
the language spoken by students in elementary and secondary schools, while 
the language of education has remained English and French. As diverse im¬ 
migrant groups have varying first languages, children of immigrants whose 
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first language is not English often start school in English 2 as either ESL, EAL, 
or ELL learners. 3 To provide assistance to these student newcomers, school 
boards and provincial ministries of education provide a supportive environ¬ 
ment through designated ELL classes in schools in addition to regular classes. 
Approaches for this language support vary from province to province, and 
the gains for students' English language proficiency development vary as 
well due to various factors influencing the language progress. 

Learning "English" and "in English" for children whose first language is 
not English has been widely researched. Collier and Thomas (1989) described 
that during the acquisition process of another language, proficiency (i.e., Eng¬ 
lish in this context) generally takes a significant number of years for newly 
immigrant students. Students of different language backgrounds who are 
newcomers in Canada may not perform as well as students who are native 
English speakers when it comes to national assessments that require certain 
reading skills. Mitchell, Destino, and Karam (1997) estimated that gaining 
proficiency in a newly learned language might take up to 10 years to attain. 
At the beginning of the learning process, the ELL student group is at a disad¬ 
vantage: Woo (2009) found that non-English Language Learners (non-ELLs) 
tend to score higher than ELLs (i.e., first-generation immigrants who are just 
learning English) in the reading assessment of National Assessment of Edu¬ 
cational Progress (NAEP), even when differences between student, teacher, 
and school backgrounds are controlled. 4 However, as time goes on, these 
ELL students whose first language is not English or who speak a variety of 
English significantly different than the type of English used for instruction 
in school (Ontario Ministry of Education, 2008) may "pick up" the language 
and eventually perform as well as their native English-speaking classmates. 
Jang, Dunlop, Wagner, Kim, and Gu (2013) found that after a certain time ELL 
students in fact outperformed the English-speaking students. 

Different factors may contribute to a student's mastery of language in dif¬ 
ferent ways. For example, the age when one immigrates seems to be a major 
factor in contributing to students' reading achievement. Recent large-scale as¬ 
sessment conducted by the Organisation for Co-operation and Development 
(OECD) recently demonstrated that 15-year-olds who have just arrived to a 
new country showed poorer reading achievement than students who immi¬ 
grated at age 5 or younger (OECD, 2013), indicating the length of residency 
and exposure to the language could be a key factor in students' success on 
their reading development. 

Although ELLs' language learning in school settings may itself be a major 
factor in contributing to students' language development, other factors 
should also be examined to get a full understanding of such development 
(Abedi 2004,2008; Abedi & Levine, 2013). For example, the amount of English 
exposure at home and at school could also be contributing factors, as vocabu¬ 
lary knowledge is associated with reading achievement (Qian, 2002; Stahl, 
Chou Hare, Sinatra, & Gregory, 1991). Family factors such as socioeconomic 
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status (SES), parental language preference, and personal literacy supports are 
also found to be related to reading achievement (D'Angiulli, Siegel, & Maggi, 
2004; Duursma et al., 2007). The level of education of the parents, especially 
the mother, is also found to be a factor contributing the reading achievement 
of students (Rabbany, Zaiane, & ElAtia, 2014). 

Many studies have been done in Canada with regard to students' read¬ 
ing achievement in various provinces. For instance, in the seminal work of 
Cummins (1981) focused on immigrants in Canada, he argued that language 
proficiency is not a unitary construct; specifically, some aspects of language 
proficiency, such as reading skills, are strongly related to cognitive and aca¬ 
demic development (p. 132). Although Abedi's (2004) work is mostly with 
Hispanic ELL students in the United States, he warns against making general¬ 
izations about the ELL group, especially the LEP (limited English proficiency) 
subgroup. Variations within the ELL group, alongside the inconsistent clas¬ 
sification of the group and the "dispersed" group, should be taken into con¬ 
sideration when interpreting the test results for making fair decisions. 

The study that we undertook for this article separates itself from others by 
looking at a large set of data for a standardized low-stakes reading achieve¬ 
ment test from across Canada, including all provinces regardless of the lan¬ 
guage situation. This test evaluated content and comprehension, and was not 
a language test per se. Our study aimed to shed some light on factors impor¬ 
tant to educators in understanding students' reading achievement within the 
multicultural context of Canada; it did not focus on one particular group. It 
further aimed to identify factors that predict students' reading performance 
on a national assessment, while taking into consideration non-native ELL and 
native English speakers. 

Purpose of the Study 

In this study, our main objective was to identify, by using Hierarchical Linear 
Modelling (HLM), factors such as ELL status that would predict students' 
reading performance in an English language arts exam from across Canada. 
We aimed to answer the following questions: 

1. What factors within students' linguistic history, language exposure, and 
practice of language contribute to predicting their reading performance? 

2. Which of these factors contribute most significantly to predicting reading 
performance? 

3. What are the effects of school variables in predicting reading perfor¬ 
mance? 

Results of this study can potentially provide new insights into factors influ¬ 
encing reading performance and competency for Grade 8 students. It may 
provide stakeholders (parents, educators, etc.) with a better understanding 
and use of assessment in order to address specific issues that relate to creat- 
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ing assistance programs for both students who are ELL and native LI English 
speakers who are at risk of low reading performance. Results in the Canadian 
context may also help educators understand how to improve teaching and 
learning support for students from diverse language backgrounds. 

Context of the Study 

This study was conducted in the Canadian context. Data from a Canadian 
national assessment, the Pan-Canadian Assessment Program (PCAP), were 
used. The PCAP is a Canada-wide achievement test that assesses performance 
of 13-year-old (Grade 8) students on their reading, mathematics, and science 
skills. The PCAP was developed by the Council of Ministers of Education, 
Canada (CMEC) in attempt to provide large-scale assessment and reporting 
on students' achievement across Canada (CMEC, n.d.). The previous form 
of PCAP was the School Achievement Indicator Program (SAIP), which was 
administered nine times between 1993 and 2004. In 2003, a decision was made 
by CMEC to replace the SAIP with the PCAP due to changes in curriculums 
and the need to reflect students' learning across Canada. The test was admin¬ 
istered using random sampling in both English and French to schools from 
across Canada. 

The PCAP has two main purposes: "(a) to help ministries and depart¬ 
ments of education make informed decisions about education, and (b) to de¬ 
termine how prepared 13-year-old students are for the academic challenges 
of high school" (CMEC, n.d., p. 1). As the entire purpose of the PCAP is to 
ensure similarity of student learning across Canada, in regards to prepara¬ 
tion for high school entry, CMEC (n.d.) stressed that results of the test should 
not be used to reflect students' individual performance, nor should results be 
used to compare schools and jurisdictions. To avoid misuse of the test results, 
the PCAP results are not available to school boards, ministries of educations, 
schools, or teachers. The PCAP is administered every three years, and each 
administration focuses on one major core subject and two minor subjects, 
alternating between mathematics, reading, and science. Along with taking 
the assessment, students are required to respond to a questionnaire target¬ 
ing detailed information such as reading habits and attitude. Additionally, a 
teacher questionnaire and a school questionnaire are also administered. 

For this study, data from the 2007 administration of the PCAP was used. 
The core subject that year was reading, with mathematics and science as the 
minor subjects. Only the data from the English administration of the read¬ 
ing assessment were used, because the interest of this study was students' 
English reading achievement. The data were obtained from all 10 provinces 
and 1 territory (Yukon) in Canada. For the English reading assessment, 1,281 
schools and 15,000 students aged 13 participated. Booklet 1 of the two read¬ 
ing booklets was used for this study; after removing missing cases, a total of 
7,467 students remained in the data for analyses. 5 The results and the answers 
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to the questionnaire provide a unique opportunity to study students' achieve¬ 
ment by looking at various elements that could contribute to progress. 

Assessing Reading: Between Language and Cognition 

Assessing reading is not an easy task. It is complex and, according to Bach¬ 
man (2000), is "the most extensively researched and most enigmatic of the 
so-called language skills" (p. x). During a reading task, two elements are 
happening: a process of reading and a product of reading—from deciphering 
the written codes and the meanings of words and sentences to understanding 
messages and connotations. Both elements are complex and involve multiple 
tasks and subtasks. Regardless of whether it is in a first language or a second/ 
foreign language, reading is a linguistic, cognitive, and metacognitive task 
that requires particular skills to happen. 

While reading is challenging for English-speaking students, the challenge 
is even higher for the non-native ELL group. It is important to understand 
the complexities of this group. The subgroups that form this larger group are 
quite different and have different results with substantial repercussions on 
the development of reading skill. 

1. The immigrant group is made up of families who chose to immigrate to 
Canada. While the main applicant must possess a higher level of educa¬ 
tion and a higher level of English competence in order to successfully 
immigrate, this is not generally the case for the accompanying spouse 
and/or children. Thus the children of these immigrant families may re¬ 
ceive very few supports at home, posing a challenge for them to perform 
well in school. Jung et al. (2013) found that the amount of exposure to 
the target culture is a critical factor to non-native ELLs. Rabbany et al. 
(2014) found that the mother's level of education plays an important role 
in achievement on the reading portion of PCAP, for both the non-native 
ELL students and regular students whose first language is English. 

2. The refugee group is entirely different from the immigrant group. In ad¬ 
dition to unpredictable levels of education and linguistic proficiency of 
their parents, students may have a gap in their prior formal education as 
well as psychologically traumatic experiences. All these factors can se¬ 
verely hinder their learning progress and negatively impact their English 
language learning process. 

The PCAP reading assessment provides a good overall view of students' 
performance as it provides not only the results of the tests, but also the sur¬ 
veys from schools, teachers, and students. Hence it gives an unique oppor¬ 
tunity to analyze the intricate skill that is reading. For Alderson (2000) there 
are two levels of understanding: one is the literal understanding of the text 
presented to the reader, and the other is the "understanding of meanings 
that are not directly stated in text... understanding of the main implications 


28 


KAREN FUNG & SAMIRA ELATIA 


of text" (p. 7). Gray (1960) made three distinctions: one linked to the literal 
meaning (reading the lines), inferring meaning from text (reading between 
the lines), and higher critical thinking meaning (reading beyond the lines). 
In light of these distinctions alone, the results of the PCAP would be directly 
influenced by the students' linguistic and educational background. We need 
to question if ELL students have matured cognitively enough to be in the 
same reading age as the other examinees. Do they possess what Grabe (1991) 
refers to as the speed and automaticity of recalling words and linked words to 
meaning? Grabe lists the following six elements necessary for a fluent reader: 
(a) automatic recognition skills, (b) vocabulary and structural knowledge, (c) 
formal discourse structure knowledge, (d) content/world background knowl¬ 
edge, (e) synthesis and evaluation skills and strategies, and (f) metacognitive 
knowledge and skills monitoring. 

From a metacognitive perspective, previous knowledge of the text or fa¬ 
miliarity with the topic of the reading text (Fisher, Frey, & Lapp, 2012) may 
play a major role in test performance. The fact that the two groups we identi¬ 
fied as ELL students (the immigrant group and the refugee group) may have 
either linguistic difficulties or educational gaps would play a major role in 
the development of the metacognitive skills necessary to understating texts. 
Abedi (2008) strongly advocates paying close attention to language profi¬ 
ciency when assessing for content among school students. In addition, items 
in the PCAP are culturally "Canadian." For instance, some items in the read¬ 
ing passages were about a game of hockey among school children. 6 Studies 
in differential item functioning found that previous knowledge of subjects 
in reading tests does influence test outcomes/performance (Recht & Leslie, 
1988). The students in our two non-native ELL groups may not be familiar 
with the "cultural" texts of the reading assessment. We wonder to what ex¬ 
tent such ELL factors impact their performance in the PCAP. 

Methodology 

Hierarchical Linear Modelling 

Flierarchy Linear Modelling was chosen as the statistical procedure in this 
study due to the nested nature of the data. FILM is related to regression anal¬ 
yses as it provides a predictive relationship between predictor variables and 
the dependent variable. One benefit of FILM over the standard regression 
analyses is that it takes into consideration the multilevel (nested) structure 
of data. When data are nested, one should not ignore that and proceed with 
regression analyses. In linear and multiple regressions, observations should 
be considered as independent of each other; however, such cases could not 
be true at all times for nested data. Snijders and Bosker (2012) explained that 
any multilevel sampling design will cause dependent observations, "and fail¬ 
ing to deal with this properly in the statistical analysis may lead to erroneous 
inferences" (p. 7). For example, students from the same classroom might be 
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more similar to each other in terms of certain characteristics in comparison 
with other students from other classes, and such similarities should not be 
ignored when performing statistical analyses due to the possible effects of 
higher-level variables on the lower level. Other predictive variables and their 
associated errors at the higher levels might also affect the entire prediction 
model; therefore, it is wiser to use the HLM when multilevel data are in¬ 
volved. In this study, a two-level HLM model was proposed in which the 
students were nested within schools. Level 1 of the model was the student's 
individual level, in which variables directly associated with the students were 
examined for their relationship with reading achievement. Level 2 was the 
school level, which was chosen because schools as units were randomly cho¬ 
sen to take the PCAP, and students in this case are nested within the schools. 
After the removal of missing cases from the Level 2 data file, a total of 1,161 
schools remained in the data for analyses. Table 1 illustrates the distribution 
of the data for schools across jurisdictions in Canada. The list of variables 
used in this study can be found in the Appendix. Examining the predictor 
variables at each of these two levels will provide a better idea of students' 
language background, foundations, and their relationships to the reading 
performance of students. The computer program HLM (Raudenbush, Bryk, 
& Congdon, 2004) was used in the analyses for this research. The dependent 
variable for the analyses was students' reading scores on the 2007 PCAP. 

Selection of Predictors for the Models 

The items on the student and school questionnaires were used to select pre¬ 
dictors to enter into the HLM model. Due to the large number of items in 
the student questionnaire, the items were chosen based on support from the 


Table 1 

School Participation by Jurisdiction 


Jurisdiction 

Number of schools participating 

Percentage in sample 

Alberta 

133 

11.5 

British Columbia 

129 

11.1 

Manitoba 

151 

13.0 

New Brunswick 

104 

9.0 

Newfoundland and Labrador 

102 

8.8 

Nova Scotia 

110 

9.5 

Ontario 

130 

11.2 

Prince Edward Island 

28 

2.4 

Quebec 

78 

6.7 

Saskatchewan 

189 

16.3 

Yukon 

7 

0.6 
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literature that such related predictors would contribute to students' achieve¬ 
ment in reading. Before entering the predictors into the model, we first cre¬ 
ated an empty model to obtain the reliability estimate of the model, and to 
determine if the sample size was appropriate for the procedure. At Level 1, 
the predictors were based on the student level; school-related variables were 
at Level 2. The student questionnaire consisted of six sections, with a total 
of 39 items (plus subquestions of A, B, C, and D). At the student level (Level 
1), several items that best represent the predictors were chosen (see Table 2). 

Questions 1 to 12 in Table 2 each represent one student individual-level 
variable. For example. Question 6 represents student's individual level of par¬ 
ticipation in class. No composite variable was used in this study. 

Table 2 

Selected Items for HLM Model at Level 1 (Student Level) 


Question Options 


1 . 

Were you born in Canada? 

Yes / No 


2. 

Have you ever been in an English-as-a-Second-Language 

Yes / No 



program or course? 



3. 

What language is most often spoken in your home? (Please 

English 

N 


check ONLY ONE RESPONSE) 

French 

N 



Canadian Aboriginal 




(e.g., Cree, Inuktitut) [ ] 



Other (e.g., German, 




Mandarin) 

N 

4. 

On average, how much time do you spend in total EACH 

Less than 30 minutes 

N 


WEEK on homework in English Language Arts? 

30 minutes to one hour 

N 



One to two hours 

N 



More than two hours 

N 

5. 

In your English Language Arts classes, how often do you 

Rarely or never 

M 


have the following kinds of assignments? 

Sometimes 

N 


Individual projects requiring work outside of class 

Often 

N 

6. 

How much does each of these apply to reading in your 

Not at all 

N 


English Language Arts classes? 

A little 

M 


1 participate in class discussions in English Language Arts 

More than a little 

N 



A lot 

N 

7. 

How often did your parents or guardians do these things 

Rarely or never 

N 


when you were younger? 

Sometimes 

N 


They encouraged me to read 

Often 

N 

8. 

When did you first learn to read? 

Before kindergarten 

N 


Kindergarten [ ] 

Grade 1 [ ] 

Grade 2 [ ] 

Later than Grade 2 [ ] 

I don’t remember [ ] 

(continued next page) 
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Question Options 


9. What language are most subjects taught in school? (Please 

English 

M 

check ONLY ONE RESPONSE) 

French 

Canadian Aboriginal 

M 


(e.g., Cree, Inuktitut) [ ] 
Other (e.g., German, 


Mandarin) 

M 

10. How would you describe your school library? 

It is a place 1 like to go to read or to study 

Yes / No 


11. Is English the language spoken at home? 7 

Yes / No 


12. Is English the language most school subjects were taught in? 

Yes / No 



At the second level (the school level), three predictors were chosen: (a) the 
percentage of ELL students in the school, (b) the size/type of community the 
school was located in, and (c) the number of minutes of instructions per week 
in English Language Arts that the school offered 13-year-olds. These variables 
were chosen based on our interest in ELL students, on whether the amount 
of English Language exposure these students receive in school matters, and 
on whether the community size of the school location matters. The associated 
questions on the school questionnaire are shown in Table 3. 


Table 3 

Selected Items for HLM Model at Level 2 (School Level) 


Question 

Options 

What percentage of students in your school are identified 

N 

0% to 25% 

as English as a Second Language (ESL 8 ) learners? 

N 

26% to 50% 


N 

51% to 75% 


M 

76% to 100%] 

In what type of community, town, or city is your school 

N 

Rural community or small town 

located? (For this question, please think of what is gener- 


(fewer than 5,000 people) 

ally considered locally to be your community, even if it is 

M 

Medium-sized town (5,000 to 

located near a larger town or city.) 


25,000 people) 


N 

Small city (25,000 to 100,000 



people) 


N 

Medium city (100,000 to 



500,000 people) 


N 

Large city (over 500,000 



people) 

How many minutes of instruction per week does your school 



offer in an English Language Arts class for 13-year-olds? 




These predictors gave an idea of how language exposure and experiences 
at the school level may predict students' reading performance. The size of 
the community where the school is located was chosen as a predictor based 
on a study by Gershenson and Langbein (2015), which indicated that school 
sizes have an impact on students' mathematics and reading achievement. 
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The larger the school's size the more negative impact it will have on students. 
Using the logic that larger schools tend to be located in larger communities, 
the community size predictor was chosen in this study. It was also believed 
that a smaller community might lead to a similarity among students' envi¬ 
ronmental background (i.e., rural or small town, and medium-sized town), 
and perhaps either a nurturing or a hindering language learning environ¬ 
ment. Slavin and Madden (1999) suggested that students might have limited 
English proficiency not only because of difficulties in learning English per se, 
but also because these students might be new immigrants living in poverty. 
They add that those who were similar in terms of language, immigration sta¬ 
tus, and socioeconomic status may tend to stay within the same community. 
Therefore it would be worthwhile to examine community size as a variable. It 
was assumed that the smaller the community, the more similar the residents 
are in comparison to larger communities, and that students' reading achieve¬ 
ment might be higher. 

All of these predictors were then entered in HLM (version 6.06) for anal¬ 
yses. Categorical responses were first dummy coded before being entered 
into HLM. Grand mean centring was used when entering variables into the 
models. Results of using HLM would help to identify significant predictors 
on students' reading achievement (for more on the chosen variables, see Ap¬ 
pendix). 

Results 

The Empty Model 

The empty model was first performed, with the following equations: 

Level-1 Model: Y ;j = p + R 

Level-2 Model: P 0 j = y 00 + u Q 

Combined Model: = y 00 + u Q + R 

Reliability estimate obtained from the empty model was 0.400, indicating a 
moderate low number. This could possibly be due to the lower number of 
students in each school after the deletion of missing cases. 

As a next step. Tables 4 and 5 were compared to determine if the sample 
size of the data was adequate. 


Table 4 

Final Estimation of Fixed Effects 


Fixed effect 

Standard 

coefficient 

Approx, error 

f-ratio 

df 

P-value 

For 

INTRCPT1, BO 






INTRCPT2, GOO 

507.576956 

1.686465 

300.971 

1113 

0.000 
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Table 5 

Final Estimation of Fixed Effects (with Robust Standard Errors) 


Fixed effect 

Standard 

coefficient 

Approx, error 

f-ratio 

df 

P-value 

For 

INTRCPT1, BO 

INTRCPT2, GOO 

507.576956 

1.686262 

301.007 

1113 

0.000 


The numbers in Tables 4 and 5 are almost exactly the same, indicating that 
the theoretical estimates were similar to the estimates obtained from the data. 
These data provided a good sample size for the model. Intraclass correlation 
calculated using sigma squared (a 2 ) and Tau (t) obtained 

Sigma_squared= 7187.4136 

Tau 

INTRCPT1,B0 = 1268.5593 

/v T 2 

P = -- 

(t 2 + a 2 ) 


The intraclass correlation demonstrated that 15.00% of the variance was over 
the student level, suggesting a multilevel nature of the data and providing a 
justification in testing additional levels of predictors. Table 6 demonstrated 
that U Q was significant, meaning that the intercepts of the schools were sig¬ 
nificantly different from each other (x 2 = 1965.43, p <.01). 


Table 6 

Final Estimation of Variance Components 



Random 

effect 

deviation 

Standard 

component 

Variance 

df 

Chi-square 

P-value 

INTRCPT1, 

u 0 

35.61684 

1268.55926 

1113 

1965.43010 

0.000 

Level-1 

R 

84.77862 

7187.41360 





Model with Added Predictors 

In this step, all the Level 1 and Level 2 predictors were entered into the model. 
Level-1 Model 

Y if Po +(V( CA N_BORN)+|3 2 *(ESL_FSL)+P 3 *(ELA_HW) + (Y(PROj_OUT) + 
P 5 *(CLASSDIS) + P 6 *(LIBRARY) + P 7 *(READPAR) + p 8 *(FIRSTREA) + 
p 9 *(LANGHOME) + P 10 *(LANGSCHE) +R ; 
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Level-2 Model 

P 0 = Yoo + Y f /( p E R CEN T ) + y 02 *(COMM_TYP)+y 03 *(MIN_INST) +u 0 

Pi = Yio 

P 2 = y 20 +Y 21 ^PERCENT) + y 22 *(COMM_TYP) 

P 3 = Y30 + Y 3 i if (MIN_INST) 

P4 = Y40 

P 5 = Y 5 o + y/(MIN_INST) 

$6 = ^60 
P7 ~ Y70 
Ps ~ ^80 

P 9 = Y 90 +Y/(COMM_TYP) 

Pi0= Y 100 + Y 101 *(PERCENT) + y 10 /(COMM_TYP) 

At Level 2, the percentage of ELL students in the school, the size/type 
of community the school was located in, and the minutes of instruction per 
week were included as predictors that would affect p n , the group mean. The 
percentage of ELLs and the type of community of the school were included 
as predictors that were believed to affect the slope of ESL_FSL (p 2 ; whether 
students have taken an ELL class). The same Level 2 variables were also as¬ 
sumed to be affecting the slope of LANGSCHE (P 10 ), or "whether English is 
the language most school subjects were taught in." The variable on the type 
of community the school is located in was assumed to be affecting the slope of 
LANGHOME (P g ), or "whether English is the language spoken at home." In 
addition, the minutes of instruction were believed to be affecting the slopes of 
ELA_HW (P 3 ; the amount of time doing English Language Art [ELA] home¬ 
work), and CLASSDIS (P 5 ; participate in class discussion). Table 7 displays 
the results of the HLM analyses. 


Table 7 

Final Estimation of Fixed Effects (with Robust Standard Errors) 

Approx. 


Standard fixed effect 

coefficient 

Error 

f-ratio 

df 

P-value 

For INTRCPT1, BO 

INTRCPT2, GOO 

492.014167 

11.278748 

43.623 

1110 

0.000 

PERCENT, G01 

9.134706 

19.071780 

0.479 

1110 

0.632 

COMM_TYP, G02 

-0.931712 

5.349020 

-0.174 

1110 

0.862 

MINJNST, G03 

-0.013325 

0.008409 

-1.585 

1110 

0.113 

For CAN_BORN slope, B1 

INTRCPT2, G10 

8.538074 

6.044707 

1.412 

4888 

0.158 


(continued next page) 
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Approx. 

Standard fixed effect coefficient Error f-ratio df P-value 


For ESL_FSL slope, B2 


INTRCPT2, G20 

PERCENT, G21 
COMM_TYP, G22 

2.196955 

-7.648031 

-0.236334 

2.402140 

6.268568 

1.647158 

0.915 

-1.220 

-0.143 

4888 

4888 

4888 

0.361 

0.223 

0.886 

For ELA_HW slope, B3 
INTRCPT2, G30 

MIN INST, G31 

2.288198 

-0.018624 

1.378027 

0.009170 

1.660 

-2.031 

4888 

4888 

0.096 

0.042 

For PROJ OUT slope, B4 

INTRCPT2, G40 

8.210754 

1.873386 

4.383 

4888 

0.000 

For CLASSDIS slope, B5 

INTRCPT2, G50 

MINJNST, G51 

14.462192 

0.007798 

1.330920 
0.005876 

10.866 

1.327 

4888 

4888 

0.000 

0.185 

For LIBRARY slope, B6 

INTRCPT2, G60 

-5.328740 

2.482299 

-2.147 

4888 

0.032 

For READPAR slope, B7 

INTRCPT2, G70 

27.215317 

1.976371 

13.770 

4888 

0.000 

For FIRSTREA slope, B8 

INTRCPT2, G80 

-9.906087 

1.331515 

-7.440 

4888 

0.000 

For LANGHOME slope, B9 

INTRCPT2, G90 
COMMJTYP, G91 

18.075113 

-0.227859 

5.760337 

3.248130 

3.138 

-0.070 

4888 

4888 

0.002 

0.945 

For LANGSCHE slope, BIO 
INTRCPT2, G100 
PERCENT, G101 

-12.876626 

-6.433678 

4.531055 

15.600035 

-2.842 

-0.412 

4888 

4888 

0.005 

0.680 

COMM TYP, G102 

7.055811 

3.532597 

1.997 

4888 

0.045 


To summarize the results, (3 Q was significant, meaning the group means 
differed significantly. The three Level 2 variables (percentage of ELL students 
in school, the type/size of community the school was located in, and the min¬ 
utes of instruction per week on ELA class) did not contribute significantly in 
making such difference (p > .05). For the Level 1 predictors, 7 out of 10 pre¬ 
dictors were significant predictors of students' reading scores (p < .05). These 
significant predictors are marked in bold in Table 7. Of all the significant 
predictors, the greatest t-ratio of READPAR suggested that parents' encour¬ 
agement to read had the greatest effect on students' reading performance. 

A positive coefficient for these predictors indicated that the higher the 
predictor's value, the higher the students' predicted reading scores. On pre¬ 
dictors that were not treated as continuous variables (i.e., LIBRARY, LANG- 
HOME, LANGSCHE), because the answer "No" was coded as "0" and "Yes" 
was coded as "1," a positive coefficient indicated that the performance score 
increased by the number of points indicated on the coefficient, on average. 
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for those who answered the question as "Yes", when other variables are 
controlled for in the model (as in the case for LANGHOME and LIBRARY). 
On the other hand, when the coefficient is negative, those who answered a 
"Yes" answer lowered the expected performance score by the points indi¬ 
cated on the coefficient, on average, when other variables are controlled for 
in the model(as in the case of LANGSCHE). In terms of Level 2 variables 
that contributed significantly in affecting the slopes of Level 1 variables, only 
MIN_INST and COMM_TYP were significant for predictors ELA_HW and 
LANGSCHE respectively. However, ELA_HW was not a significant predictor 
of student scores. 

The combined model with all the predictors is as follows: 

Y ii = (too + Y„i*(PERCENT) + Y 02 *(COMM_TYP) + y 03 *(MIN_INST) +u 0 ) + 
[y 10 (CAN_BORN)] + [[y 20 + y 21 *(PERCENT) + y 22 *(COMM_TYP)]* 
(ESL_LSL)] + [[y 30 + y 31 *(MIN_INST)]*(ELA_HW)] + 

[ Y 40 (P ROJ_OUT) ] + [[y 50 + (Y 51 WN_INST)]*(CLASSDIS)] + 
[Y 60 (LIBRARY)] + [y 70 (READPAR)]+ [y 80 (PIRSTREA)]+ [[y 90 + 
Y 91 *(COMM_TYP) ] *(L AN GHOME) ] + [[y 100 + y 101 *(PERCENT) + 
Y 102 *(COMM_TYP)]*(LANGSCHE)] + R 

The combined model with only significant predictors is as follows: 

Y ii = too + [Y 4 o(P ROJ_OUT) ] + [(y 50 (CLASSDIS)] + [y 60 (LIBRARY)] + 
[Y 70 (READPAR)]+ [y 80 (FIRSTREA)]+ [y 90 (LANGHOME)] + 

[[Y100+ Yio2*(COMM_TYP)]*(LANGSCHE)] + R 

Explained Variance 

Total variability from empty model = x 2 + a 2 

= 1268.55926+7187.41360 
= 8455.97 

Variability after predictors were added: 

= 946.96966 + 6452.09659 
= 7399.07 

Percentage of explained variance = 7399.07 / 8455.97 

= 0.88 
= 1 - 0.88 
= 0.12 -» 12 % 

The percentage of explained variance suggested that the Level 1 predicting 
variables contributed 12% to the model. 
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Discussion and Conclusion 

This research utilized HLM to identify the predictive relationship between 
students' linguistic history, language exposure, and practice of language on 
reading performance. Using the 2007 PCAP data, results demonstrated that 
at the student level, the number of individual projects requiring students to 
work outside of class, student's participation in class discussion during class, 
English being the language spoken at home (which was positively correlated 
with reading achievement), how often the parents encourage the student to 
read, age a child first learned to read, whether students like to use the library 
as a place for study or reading, and whether English is the language most 
school subjects were taught in were found to be predictors of statistical sig¬ 
nificance. These predictors contributed 12% of the variability in the model, 
and how often parents encouraged students to read had the greatest effect of 
all significant predictors on reading achievement. It was surprising that ELL 
did not contribute significantly as a predictor of students' reading scores. 
However, it should be noted that this variable did not take into consideration 
how long the students had been enrolled into the ELL classes. The duration 
which students were taking ELL classes may contribute on different levels of 
reading skills. 

At Level 2, all three predictors (i.e., percentage of ELL students in school, 
the type/size of community the school located in, and the minutes of instruc¬ 
tion per week on ELA) were found to be insignificant in predicting students' 
reading scores. And since the intraclass correlation indicated that having a 
second (school) level in the analyses was necessary, the results of this study 
calls for selection of better Level 2 predictors in future research. 

The results of this study provided useful information regarding the ex¬ 
tent to which students' language exposure and experiences contributed to 
predicting their reading achievement. However, one should note that only 
about 12% of the variability was accounted for by these variables, indicating 
a possible underrepresentation of students' language profile. More significant 
predictors should be added in order to get a full picture, especially at Level 
2, to better predict students' reading achievement. Luture research may look 
at other factors outside of students' language background, for example, the 
socioeconomic status of families, mother's level of education, the province in 
which the school is located, or the size of classrooms. The result of low vari¬ 
ability might also be due to the lack of representation of the predictors by the 
chosen items. Because only one survey item was selected to represent one cat¬ 
egory of predictor in this study, future research may group the items together 
based on categories to create composite variables, and obtain a category score 
representing the mean of the grouped items together. This step would en¬ 
sure that the majority of the survey items are considered when entering data 
into the model, creating a better representation of the predictor. In regards 
to our interest in the ELL group, although this study has found that being 
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in ELL classes did not contribute significantly to predicting students' read¬ 
ing performance, as discussed above, factors such as duration of students' 
enrolment in ELL class, and for non-Canadian-bom students the age they 
moved to Canada and the amount of support received within and outside of 
their families, could be factors worth examining. In this study we have only 
examined the effects on reading performance; we do not know how the same 
variables may affect writing, which always goes hand in hand with reading 
in terms of language assessment. Luture studies may also look at the writing 
outcomes of students. 

The use of a Canadian performance measure (the PCAP) allowed results 
to be applied in a multicultural framework. It could further help various 
stakeholders in assisting students to improve their reading abilities, espe¬ 
cially for ELL students who are newcomers to an English-speaking coun¬ 
try. In addition, parents and teachers could benefit from these results and 
improve students' reading environments and habits. Lor example, teachers 
may create more targeted opportunities for students to practice their reading 
skills, hence allowing them more exposure for gaining reading experiences 
within the school. Parents and/or guardians of ELL students may also imple¬ 
ment strategies at home for encouraging reading in English in order to help 
their children with English reading proficiency, and for better understanding 
of language learning processes within contents and subject areas. Although 
more predictors have yet to be identified, this study serves as a first step in 
understanding the factors on predicting students' reading achievement. 

Luture research could also investigate the age of the students taking the 
assessment as another important factor for consideration. Such studies would 
be consistent with the results and conclusions of Cummins (1981) in regards 
to whether or not the age of ELL students is a contributing factor to their 
reading performance. Students at this age, 13-years-old, are at a critical pe¬ 
riod for learning, from the perspective of both second language acquisition 
(SLA) theory and cognitive psychological development. In order to advance 
both fields in understanding how reading as a literacy skill intertwines with 
language acquisition, a longitudinal study spanning three years—from a year 
before taking the PCAP to two years after sitting for the PCAP—may contrib¬ 
ute to greater understanding of the effect of age. 
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Notes 

1 The corresponding author and project leader of this study. 

2 For the purpose of this article, French and francophone immigrants are not included in this 
study. 

3 School-age children in K-12 who are learning English in an ESL or EAL setting are com¬ 
monly referred to as ELL — English Language Learner/Learning (Abedi & Gandara, 2007; Abedi, 
Hofstetter, & Lord, 2004). This distinguishes them from students learning English as a language 
art subject who speak English at home as their first/native language. English is not the language 
ELL students identify with outside school, and they have varying degrees of competence in it. 
In this article, and for saliency purposes, we will be using ELL for students who are learning 
English but are not native speakers of English; to them English could be a second language, or it 
could be an additional language to one or more languages already spoken at home. 

4 "ELL students" refers to students in K-12 schooling as opposed to ELL adults. 

5 The First Nations group was not being studied in this article due to its small sample size in 
the PCAP data. 

6 We cannot share real test items as we are bound by a confidentiality agreement with CMEC. 

7 Due to the amount of dummy coding required for certain categorical variables, which would 
result in a higher number of additional predictors, the authors have changed the forms of the 
questions into dichotomous responses. 

8 The term "ESL" here is copied directly from the PCAP documents; we opted to keep the 
questions as they appear in the documentation. It is used as such when we ran the analyses as 
well. 
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Appendix. Representation of Variable Names 


Question 

Variable Name 

Were you born in Canada? 

CAN_BORN 

Have you ever been in an English-as-a-Second-Language program or 
course? 

ESL_FSL 

What language is most often spoken in your home? (Please check ONLY 
ONE RESPONSE) 

LANGHOME 

On average, how much time do you spend in total EACH WEEK on home¬ 
work in English Language Arts? 

ELA_HW 

In your English Language Arts classes, how often do you have the following 
kinds of assignments? Individual projects requiring work outside of class 

PROJJDUT 

How much does each of these apply to reading in your English Language 
Arts classes? 1 participate in class discussions in English Language Arts 

CLASSDIS 

How often did your parents or guardians do these things when you were 
younger? They encouraged me to read 

READPAR 

When did you first learn to read? 

What language are most subjects taught in school? (Please check ONLY 
ONE RESPONSE) 

FIRSTREA 

How would you describe your school library? It is a place 1 like to go to read 
or to study 

LIBRARY 

What percentage of students in your school are identified as English as a 
Second Language (ESL) learners? 

PERCENT 

In what type of community, town, or city is your school located 7(For this 
question, please think of what is generally considered locally to be your 
community, even if it is located near a larger town or city.) 

COMM_TYP 

How many minutes of instruction per week does your school offer in an 
English Language Arts class for 13-year-olds? 

MINJNST 
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